Derman's book as inspiration: some results on LP for MDPs
نویسنده
چکیده
In 1976 I was looking for a suitable subject for my PhD thesis. My thesis advisor Arie Hordijk and I found a lot of inspiration in Derman’s book (Finite state Markovian decision processes, Academic Press, New York, 1970). Since that time I was interested in linear programming methods for Markov decision processes. In this article I will describe some results in this area on the following topics: (1) MDPs with the average reward criterion; (2) additional constraints; (3) applications. These topics are the main elements of Derman’s book.
منابع مشابه
Symmetric Primal-Dual Approximate Linear Programming for Factored MDPs
A weakness of classical Markov decision processes is that they scale very poorly due to the flat state-space representation. Factored MDPs address this representational problem by exploiting problem structure to specify the transition and reward functions of an MDP in a compact manner. However, in general, solutions to factored MDPs do not retain the structure and compactness of the problem rep...
متن کاملComparative Evaluation of the Efficacy of Wisdom and Inspiration in Abu Hatam’s and Zakariyya Razi’s Opinions
The question has always been raised throughout history whether human beings need to follow the prophets or divine revelation to achieve salvation. There will be no need, as some believe, to follow divine teachings once human beings reach intellectual maturity; however, others insist on the permanent need for Guidance from God due to inadequacy of human reason. In the third or fourth century AH,...
متن کاملAccelerated decomposition techniques for large discounted Markov decision processes
Many hierarchical techniques to solve large Markov decision processes (MDPs) are based on the partition of the state space into strongly connected components (SCCs) that can be classified into some levels. In each level, smaller problems named restricted MDPs are solved, and then these partial solutions are combined to obtain the global solution. In this paper, we first propose a novel algorith...
متن کاملA Linear Programming Approach to Nonstationary Infinite-Horizon Markov Decision Processes
Nonstationary infinite-horizon Markov decision processes (MDPs) generalize the most well-studied class of sequential decision models in operations research, namely, that of stationaryMDPs, by relaxing the restrictive assumption that problem data do not change over time. Linearprogramming (LP) has been very successful in obtaining structural insights and devising solutionmeth...
متن کاملEfficient Solution Algorithms for Factored MDPs
This paper addresses the problem of planning under uncertainty in large Markov Decision Processes (MDPs). Factored MDPs represent a complex state space using state variables and the transition model using a dynamic Bayesian network. This representation often allows an exponential reduction in the representation size of structured MDPs, but the complexity of exact solution algorithms for such MD...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Annals OR
دوره 208 شماره
صفحات -
تاریخ انتشار 2013